Information processor having input system using stroboscope

ABSTRACT

A stroboscope is utilized as an input device of a golf game system ( 10 ), for example. The golf game system includes a game machine ( 12 ) as an information processing apparatus and a golf-club-shaped input device ( 14 ), and within a housing of the game machine, an imaging unit ( 28 ) is housed, and the imaging unit is provided with an image sensor ( 40 ) and an infrared-LED. By utilizing the infrared-LED, an infrared ray is intermittently emitted to a predetermined range of an upper portion of the imaging unit. Accordingly, the image sensor intermittently images a reflective body provided in the golf-club-shaped input device moving within the range. Such the stroboscope image processing of the reflective body enables calculation of a velocity, and so on as an input of the game machine.

CROSS-REFERENCE TO RELATED APPLICATION

This application is a National Stage entry of International ApplicationNo. PCT/JP03/07900, filed Jun. 20, 2003, the entire specification claimsand drawings of which are incorporated herewith by reference.

TECHNICAL FIELD

The present invention relates to an information processing apparatusprovided with an input system utilizing a stroboscope. Morespecifically, the present invention relates to an information processingapparatus that processes an image signal of an object irradiated by thestroboscope. Furthermore, the present invention relates to anentertainment apparatus such as a game apparatus, etc. provided with aninput system utilizing a stroboscope. In addition, the present inventionrelates to a man-machine interface system provided with an input systemutilizing a stroboscope.

PRIOR ART

The applicant of this invention proposes that in the Patent Literature1, and so on sensory video game machines in which change in accelerationof a bat or a racket in a real space is detected by use of apiezoelectric buzzer so as to be used as a game input. In such thesensory video game machines, when the change in acceleration is equal toor more than a predetermined amount, it is determined that the gameplayer operates (swings) an object (bat or racket in the above-describedexample) in the real space.

Patent Literature 1: Japanese Patent Laying-open No. 2001-104636

Patent Literature 2: Japanese Patent Laying-open No. 2002-231489

Patent Literature 3: Japanese Patent Laying-open No. 7-141101

However, depending on the kind of the game, information indicative ofwhich position, which velocity, or which direction the operation isperformed as well as whether or not the object is performed, that is,whether or not the acceleration is applied to the object may berequired. In the input device disclosed in the Patent Literature 1, itis impossible to fulfill such the requirement.

On the other hand, if imaging the object by utilizing the stroboscopedisclosed in the Patent Literature 2, for example, by analyzing an imagesignal, the position and the velocity of the object as described abovecan be obtained. However, in the Patent Literature 2, the stroboscope isonly disclosed, and the Patent Literature 2 does not provide a specificmethod of imaging an object by use of the stroboscope, and analyzing inreal time an image signal obtained through it.

It is noted that the Patent Literature 3 discloses that an object isextracted from an imaged image signal to obtain the position of theobject, and the position information is used as an input of gameapparatuses or computers. However, this method works well in a specificuse environment, but it is very difficult to obtain accurate positioninformation in a room at a general home where game machines are used.This is because all of the illuminations in the room, windows, objectsin various colors, and moving objects except for the game player exertan influence as noise and disturbance on the detection accuracy. Inorder to accurately detect the position of the object with the influenceof the noise and the disturbance reduced, a high-speed processor isnecessary, and this is unrealistic for the information processingapparatus with restricted processing ability and at low costs.

SUMMARY OF THE INVENTION

Therefore, it is a primary object of the present invention to provide anovel information processing apparatus, an entertainment apparatus, anda man-machine interface system that are able to apply an input in realtime to a computer or a game machine by use of a stroboscope.

The present invention relates to an information processing apparatusprovided with an input system utilizing a stroboscope, and comprises astroboscope; an imaging means for imaging an object at a light-emissionand at a non-light-emission of the stroboscope to output an image signalat light-emission and an image signal at non-light emission; a firstmeans for calculating a part or all of information of a position, asize, a velocity, an acceleration, a moving path pattern of the objecton the basis of differences between the plurality of image signals atlight-emission and the plurality of image signals at non-light emission;and a second means for performing information processing on the basis ofthe information calculated by the first means.

The above-described first means may include a determination means fordetermining whether or not the information is coincident with apredetermined condition.

In one embodiment, the first means includes a valid input detectingmeans for detecting only valid information out of the information on thebasis of the determination result by the determination means, andtransmitting to the second means as the valid information beingperformed.

Furthermore, the first means includes a distance calculating means forcalculating a distance between the object and the imaging means from theinformation indicative of the size of the object.

In addition, the first means includes an analyzing means for analyzinginformation obtained from a difference between the image signal atlight-emission and the image signal at non-light emission to extract theshape of the object, and an angle calculating means for calculating anangle between the object and the imaging means from the shape.

The analysis by the above-described analyzing means is for extractingpredetermined two points within the object, and the calculation of theangle by the angle calculating means is for calculating an angle betweena line segment between predetermined two points and a predeterminedcoordinate axis.

In an embodiment, a time interval of the light-emission of thestroboscope is freely configurable.

Furthermore, a length of the light-emission period and a length of thenon-light-emission period of the stroboscope are freely configurable.

In addition, an exposure period of the imaging means is freelyconfigurable.

In one embodiment, the object includes a reflective body.

Furthermore, the stroboscope includes a light source for outputting alight having a specific wavelength range, and the imaging means is madeto respond to only the specific wavelength range.

In this case, the imaging means includes a filter for passing only thelight with the specific wavelength range and an imaging device forimaging an image formed by the light passed through the filter.

In another embodiment, the imaging means includes an imaging device forimaging only the image formed by the light having the specificwavelength range.

Each of the first means and the second means described above is processto be processed by a single or a plurality of processors.

Then, the image processing performed by such the second means is anentertainment process such as a game, etc.

A man-machine interface system provided with an input system byutilizing a stroboscope comprises a stroboscope; an imaging means foroutputting an image signal at a light-emission and an image signal at anon-light emission by imaging an object at a light-emission of thestroboscope and at a non-light-emission; a first means for calculating apart or all of information of a position, a size, a velocity, anacceleration, a moving path pattern on the basis of differences betweenthe plurality of image signals at a light-emission and a plurality ofimage signals at a non-light emission; and a second means for performingimage processing on the basis of the information calculated by the firstmeans.

According to the invention, by the stroboscope (42, 52: referencenumerals of elements or components corresponding in the embodiments, andso forth), the object (14, 94, 112) is brightly irradiated to heighten acontrast between the object and things except for the object as animaging result, and therefore, it becomes easy to detect the object.Furthermore, the first means (52, S59, S129, FIG. 35: S61, FIG. 22, FIG.25, FIG. 33) calculates the differences between the plurality of imagesignals at light-emission and the plurality of image signals atnon-light emission, and whereby, it becomes possible to detect theposition, the size, the velocity, the moving path pattern of the objectwith accuracy and with simple image processing while reducing aninfluence of noise component due to a still image except for the objectbeing a moving object, a fixed light source, and etc. On the basis ofthe information thus calculated, the second means (52, S63) performs thepredetermined information processing.

Then, by separating the information processing for calculating theseinformation and the information processing at the application side, itis possible to make the information processing at the application sidesimple, and at a time of replacing the information processing at theapplication side with another processing, the processing for calculatingthe position, the size, the velocity, the acceleration, the moving pathpattern of the object can be utilized without being changed.

In a case of utilizing the determination means, the determination means(52, S61, FIG. 22, FIG. 25, FIG. 33) determines whether or not theposition, the size, the velocity, the acceleration, the moving pathpattern of the object are coincident with the predetermined condition,and in the information processing at the application side, withreference to the determination result, if the information is notcoincident with the predetermined condition, the calculated informationis not received, capable of performing simple application processing.

More specifically, the valid input detecting means (52, FIG. 22, FIG.25, FIG. 33) included in the first means makes a choice of theinformation on the basis of the determination result by thedetermination means, and transmits only the valid input from the user tothe information processing at the application side. Therefore, itbecomes possible to perform simple application processing.

In a case of utilizing the distance calculating means, the distancecalculating means (52, S111, S113) included in the first meanscalculates the size of the object from the imaging result, andcalculates the distance between the object and the imaging means on thebasis of the calculated size information. Thus, from the two-dimensionalimaging result, the position, the velocity, the acceleration, and themoving path pattern in the three-dimensional space can be obtained.

In a case that the first means includes the analyzing means (52, FIG.24: S159-S167) and the angle calculating means (52, S169), by analyzingthe shape of the object from the imaging result, the angle formed by theobject projected on the two-dimensional image as the imaging result andthe imaging means can be obtained.

The analyzing means performs the analysis (S166) for extracting the twopredetermined points within the object, and the angle calculationexecuted by the angle calculating means is for calculating the anglebetween the line segment between the predetermined two points and thepredetermined coordinate axis.

For example, as shown in FIG. 6 embodiment, by controllinglighting/lights-out of the infrared-LED by the processor, it becomespossible to perform the light-emission of the strobe light source andthe exposure by the imaging means only at a necessary time interval andat a necessary timing, capable of reducing electric power consumption.

If the object includes the reflective body (50, 50A, 100, 116), acontrast between the object and other images are further enhanced,capable of improving accuracy of the detection with an inexpensiveconfiguration.

In a case of responding to the specific wavelength, the stroboscopeincludes the light source (the infrared-LED 42, for example) outputtingthe light having the specific wavelength range, and the imaging meansresponds only to the specific wavelength range by utilizing the infraredfilter. Therefore, rendering the light having the wavelength that themoving light source and blinking light source (fluorescent lamp, etc.)except for the object to be detected do not have the light source of thestroboscope, it is possible to remove the noise light sources.

In the embodiment, each of the first means and the second means is aprocess to be processed by a single or a plurality of processors(processors for processing 52 and/or 63). Thus, the first means and thesecond means are rendered processes to be processed as software of theprocessor, capable of establishing a system with low prices and withhigh flexibility. It is noted that it is further desirable that theprocesses of both of the first means and the second means are executedby a single processor.

Then, the image processing performed by the second means is anentertainment process such as a game, etc.

By utilizing the man-machine interface according to this invention as aman-machine interface for a personal computer, a workstation, gameequipment, educational equipment, medical equipment, etc., it ispossible to establish the input system with low prices and with highdegree of precision.

According to this invention, by digitally analyzing the imaging resultof the object irradiated by the stroboscope, the information such as theposition, the size, the velocity, the acceleration, the moving pathpattern of the object can be used as inputs to the informationprocessing apparatus.

Furthermore, with simple information processing, it is possible toperform detection independent of the noise or the disturbance with highdegree of precision, and therefore, it becomes possible to use on asystem restricted in performance depending on the condition such asprices, allowable electric power consumption, etc.

The above described objects and other objects, features, aspects andadvantages of the present invention will become more apparent from thefollowing detailed description of the present invention when taken inconjunction with the accompanying drawings.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is an illustrative view showing one example of a strobe imageimaged by a stroboscope;

FIG. 2 is an illustrative view showing an entire configuration of a golfgame system of one embodiment of the present invention;

FIG. 3 is an illustrative view showing one example of an imaging unit ofFIG. 2 embodiment;

FIG. 4 is an illustrative view showing one example of a golf-club-shapedinput device of FIG. 2 embodiment;

FIG. 5 is a block diagram showing FIG. 2 embodiment;

FIG. 6 is a schematic circuit diagram showing a configuration offetching pixel data into a game processor from an image sensor and anLED driving circuit in FIG. 5 embodiment;

FIG. 7 is a timing chart showing an operation of FIG. 6 embodiment;

FIG. 8 is a timing chart showing a part of FIG. 7 in an enlarged manner;

FIG. 9 is an illustrative view showing a state or state transition ofFIG. 2 embodiment;

FIG. 10 is a flowchart showing an entire operation of FIG. 2 embodiment;

FIG. 11 is a flowchart showing a sensor initializing process being oneexample of an initialization process of FIG. 10 embodiment;

FIG. 12 is a flowchart showing a command transmitting process in FIG.11;

FIG. 13 is a flowchart showing a register setting process in FIG. 11;

FIG. 14 is a timing chart showing the register setting process shown inFIG. 13;

FIG. 15 is a flowchart showing an operation of a game processor in FIG.2 embodiment;

FIG. 16 is a flowchart showing a stroboscope imaging process in FIG. 15embodiment;

FIG. 17 is a flowchart showing a pixel data fetching process in FIG. 15embodiment;

FIG. 18 is a flowchart showing a notable point extracting process inFIG. 15 embodiment;

FIG. 19 is an illustrative view showing a principle adopting differencedata in embodiments;

FIG. 20 is a flowchart showing a velocity vector calculating process inFIG. 15 embodiment;

FIG. 21 is an illustrative view showing respective coordinates positionsin the velocity vector calculating process;

FIG. 22 is a flowchart showing a determination process in FIG. 15embodiment;

FIG. 23 is an illustrative view showing a modified example of thegolf-club-shaped input device utilized in the golf game system;

FIG. 24 is a flowchart showing the notable point extracting processingoperation in a case of utilizing the golf-club-shaped input device inFIG. 23;

FIG. 25 is a flowchart showing a determination process in FIG. 15embodiment in a case of utilizing the golf-club-shaped input device inFIG. 23;

FIG. 26 is an illustrative view showing an angle in the determinationprocessing operation in FIG. 25;

FIG. 27 is an illustrative view showing an entire configuration of abowling game system of another embodiment of the present invention;

FIG. 28 is a sectional illustrative view showing an internalconfiguration of a ball-shaped input device shown in FIG. 27 at a lineXXVIII-XXVIII;

FIG. 29 is an illustrative view showing one example of a game screen tobe displayed on a television monitor in FIG. 27 embodiment;

FIG. 30 is an illustrative view showing one example of a score sheet tobe displayed on the television monitor in FIG. 27 embodiment;

FIG. 31 is a block diagram showing FIG. 27 embodiment;

FIG. 32 is a flowchart showing an entire operation of FIG. 27embodiment;

FIG. 33 is a flowchart showing a determination process in FIG. 15embodiment in the bowling system in FIG. 27;

FIG. 34 is an illustrative view showing one example of a glove-shapedinput device;

FIG. 35 is a flowchart showing a movement detecting processing operationshown in FIG. 15 embodiment in a case of utilizing the glove-shapedinput device in FIG. 34; and

FIG. 36 is a flowchart showing the pixel data array fetching process inFIG. 15 embodiment.

BEST MODE FOR PRACTICING THE INVENTION

With referring to FIG. 2, a golf game system 10 of one embodiment of thepresent invention includes a game machine 12 and a golf-club-shapedinput device 14, and the golf-club-shaped input device 14 is swung by agame player over the game machine 12. It is noted that the game machine12 is driven by a direct-current power supply such as an AC adapter (notshown) or a battery, for example. The game machine 12 is furtherconnected to an AV terminal (not shown) of a television monitor (notshown) via an AV cable 16.

The game machine 12 includes a housing 18, and on the housing 18, apower switch 20 is provided, and a direction button 22, a determinationkey 24 and a cancel key 26 are also provided. The direction button 22has respective buttons of four directions (a up, down, left and right),and utilized for moving a cursor in order to select a menu or a gamemode on a display screen of a television monitor, for example. Thedetermination key 24 is utilized for determining an input to the gamemachine 12. Furthermore, the cancel key 26 is utilized for canceling aninput to the game machine 12.

Inside the housing 18 of the game machine 12, an imaging unit 28 shownin FIG. 3 in detail is housed. The imaging unit 28 includes a unit base30 formed by a plastic molding, for example, and to the unit base 30, asupporting cylinder 32 is attached. On an upper surface of thesupporting cylinder 32, an opening 34 whose inside is in a form ofinverted-conical shaped bugle is formed, an optical system including aconcave lens 36 and a convex lens 38 both formed by a transparentplastic molding, for example, is provided inside a cylindrical portionbelow the opening 34, and beneath the convex lens 38, an image sensor 40as an imaging device is fixed. Accordingly, the image sensor 40 canphotograph an image corresponding to a light incident from the opening34 through the lenses 36 and 38.

The image sensor 40 is a CMOS image sensor of low resolution (forexample, 32×32 pixels: gray scale). It is noted that the image sensor 40may have pixels greater in number, or may consist of another device suchas CCD, or the like.

Furthermore, the unit base 30 is provided with a plurality ofinfrared-LEDs (four (4) in this embodiment) each having an upper lightemitting direction. By the infrared-LEDs 42, an infrared light isirradiated within a range defined by the lines 44 a and 44 b above theimaging unit 28 shown in FIG. 3. Furthermore, above the unit base 30, aninfrared filter (filter for passing through only the infrared ray) isattached so as to cover the above-described opening 34. Then, theinfrared-LED 42 is as described later continuously turned on/off so asto function as an infrared stroboscope. It is noted that the“stroboscope” is a generic name for intermittently irradiating a movingbody. Accordingly, the above-described image sensor 40 images a movingbody, that is, a reflective body 50 (see FIG. 4) provided on thegolf-club-shaped input device 14 in this embodiment, which moves withina imaging range shown by lines 44 a and 44 b.

The golf-club-shaped input device 14 is entirely formed by a plasticmolding, for example, and, as shown in FIG. 4, includes a club shaft 46and a club head 48 attached at a tip end of it. At the bottom of theclub head 48, a circle-shaped reflective body 50 consisting of aretroreflection sheet, for example, is provided. Accordingly, as shownin FIG. 2, when a game player swings with the club shaft 46 as the inputdevice 44 in his hands above the game machine 12 as he generally playsthe golf, a light reflected by the reflective body 50 at the bottomsurface of the head 48 is imaged by the image sensor 40. At this time,the infrared-LED 42 intermittently irradiates the infrared light asdescribed above, and therefore, the reflective body 50 is intermittentlyimaged as shown in FIG. 3. In the golf game system 10 of thisembodiment, as described later, by such the strobe image processing ofthe reflective body, a velocity as an input to the game machine 12, etc.is calculated.

With referring to FIG. 5, the club-shaped input device 14 is irradiatedby a light emission from the infrared-LED 42, and reflects the infraredlight by the reflective body 50. The reflected light from the reflectivebody 50 is imaged by the image sensor 40, and the image sensor 40outputs an image signal of the reflective body 50. The analog imagesignal from the image sensor 40 is converted into digital data by an A/Dconverter (not shown) included in a game processor 52.

It is noted that the game processor 52 intermittently blinks theinfrared-LED 42 for the above-described strobe imaging.

Although arbitrary kinds of processor can be utilized as such the gameprocessor 52, a high-speed processor developed by the applicant of thepresent invention and already filed as a patent application is used inthis embodiment. This high-speed processor is disclosed in detail inJapanese Patent Laying-open No. H10-307790 [G06F13/36, 15/78] and U.S.Pat. No. 6,070,205 corresponding thereto.

Although not illustrated, the game processor 52 includes variousprocessors such as a CPU, a graphics processor, a sound processor, a DMAprocessor, etc. and also includes the above-described A/D converter usedfor fetching an analog signal, and an input/output control circuit forreceiving an input signal such as a key operation signal and an infraredsignal, and giving an output signal to external equipment. Therefore, aninput signal from the operation keys 22-26 is applied to the CPU throughthe input/output control circuit. The CPU executes a required operationaccording to the input signal, and applies the result to the graphicsprocessor, the sound processor, etc. Therefore, the graphic processorand the sound processor execute an image processing and a soundprocessing according to the operation result.

Although not illustrated, the processor 52 is provided with an internalmemory, and the internal memory includes a ROM or a RAM (SRAM and/orDRAM). The RAM is utilized as a temporary memory, a working memory, acounter, or a register area (temporary data area), and a flag area. Itis noted that the processor 52 is connected with a ROM 54 via anexternal bus. In the ROM 54, a game program described later is set inadvance.

The processor 52 processes a digital image signal input from the imagesensor 40 via the A/D converter to detect a movement of thegolf-club-shaped input device 14, and executes an arithmetic operation,a graphic processing, a sound processing, etc. to output a video signaland an audio signal. The video signal is an image signal for displayinga game screen, and the audio signal is a signal for a game music and asound effect. Accordingly, a game screen is displayed on the screen of atelevision monitor (not shown), and a necessary sound (sound effect,game music) is output from its speaker.

Here, with referring to FIG. 6 to FIG. 8, a configuration for fetchingpixel data to the game processor 52 from the COMS image sensor 40 isdescribed in detail. As shown in FIG. 6, the CMOS image sensor 40 inthis embodiment is a type of outputting a pixel signal as an analogsignal, and therefore, the pixel signal is input to an analog input portof the game processor 52. The analog input port is connected to an A/Dconverter (not shown) within the game processor 52, and thus, the gameprocessor 52 obtains or fetches from the A/D converter the pixel signal(pixel data) that has been converted to digital data.

A midpoint of the above-described analog pixel signal is determined by areference voltage applied to a reference voltage terminal Vref of theCMOS image sensor 40. Therefore, in this embodiment, as to the imagesensor 40, a reference voltage generating circuit 56 including aresistor voltage-dividing circuit, for example, is provided, and fromthe reference voltage generating circuit 56, a reference voltage havinga constant magnitude is always applied to the reference voltage terminalVref.

Each of digital signals for controlling the CMOS image sensor 40 isapplied to the I/O port of the game processor 52, or output therefrom.The I/O ports are digital ports capable of controlling respectiveinput/outputs, and connected to an input/output control circuit (notshown) within the game processor 52.

More specifically, from the output port of the game processor 52, areset signal for resetting the image sensor 40 is output so as to beapplied to the image sensor 40. Furthermore, from the image sensor 40, apixel data strobe signal and a frame status flag signal are output, andthese signals are applied to the input port of the game processor 52.The pixel data strobe signal is a strobe signal as shown in FIG. 7( b)for reading each of the pixel signals described above. The frame statusflag signal is a flag signal for showing a state of the image sensor 40,and defines an exposure period of the image sensor 40 as shown in FIG.7( a). A low level of the frame status signal shown in FIG. 7( a)indicates an exposure period, and a high level thereof shown in FIG. 7(a) indicates a non-exposure period.

Furthermore, the game processor 52 outputs from the I/O port a command(or command+data) set in a control register (not shown) within the CMOSimage sensor 40 as register data, and outputs a register setting clockin which the high level and the low level are repeated and applies it tothe image sensor 40.

It is noted that as the infrared-LED 42, four (4) infrared-LEDs 42 a, 42b, 42 c, and 42 d connected in parallel with each other are utilized asshown in FIG. 6. These four (4) infrared-LEDs 42 a-42 d are arrangedsuch that they irradiate the object (golf-club-shaped input device 14)as described above, emits an infrared light in a direction the same asviewpoint direction of the image sensor 40, and surrounds the imagesensor 40. It is noted that these respective infrared-LEDs 42 a-42 d aresimply called as the infrared-LED 42 except as especially required bydiscrimination. The infrared-LED 42 is turned on and is extinguished(turned off) by the LED driving circuit 58. The LED driving circuit 58receives the above-described frame status flag signal from the imagesensor 40, and the flag signal is applied to a base of a PNP transistor68 through a differentiation circuit 60 consisting of a resistor 62 anda capacitor 64. The PNP transistor 68 is connected with a pull-upresistor 66, and the base of the PNP transistor 68 is normally pulled upto a high level. Then, when the frame status signal becomes a low level,the low level is input to the base through the differentiation circuit60, and the PNP transistor 68 is turned on only during a low levelperiod of the flag signal.

A collector of the PNP transistor 68 is grounded via resistors 70 and72. Then, a node of collector resistances 70 and 72 is connected to thebase of an NPN transistor 74. A collector of the NPN transistor 74 iscommonly connected to anodes of the respective infrared-LEDs 42 a-42 d.An emitter of the NPN transistor 74 is directly connected to a base ofanother NPN transistor 76. A collector of the NPN transistor 74 iscommonly connected to cathodes of the respective infrared-LEDs 42 a-42d, and the emitter thereof is grounded.

In the LED driving circuit 58, the infrared-LED 42 is turned on duringonly a period when the LED control signal (corresponding to a secondsignal) output from the I/O port of the game processor 52 is active(high level), and the frame status flag signal from the image sensor 40is a low level. As shown in FIG. 7( a), when the frame status flagsignal becomes the low level, the PNP transistor 68 is turned on duringthe low level period (although there is a delay due to the time constantof the differentiation circuit 60 in reality). Accordingly, when the LEDcontrol signal shown in FIG. 7( d) is output from the game processor 52at the high level, the base of the NPN transistor 74 becomes a lowlevel, and then, the transistor 68 is turned off. When the transistor 68is turned off, the transistor 74 is turned on. Accordingly, a currentflows a power supply (shown by a small white circle in FIG. 6) throughthe respective infrared-LEDs 42 a-42 d and the transistor 76, and inresponse thereto, the respective infrared-LEDs 42 a-42 d are turned onas shown in FIG. 7( e).

In the LED driving circuit 58 of this embodiment, the infrared-LED 42 isturned on only when the LED control signal shown in FIG. 7( d) isactive, and the frame status flag signal is low level, and therefore,the infrared-LED 42 is turned on only during an exposure period (seeFIG. 7( f)) of the image sensor 40. Therefore, according to thisembodiment, it is possible to reduce useless power consumption.Furthermore, the frame status flag signal is coupled by the capacitor64, and therefore, on assumption that the flag signal is suspended withthe low level kept due to a hang-up of the image sensor 40 and the like,the transistor 68 is surely turned off after a lapse of a predeterminedtime period, and the infrared-LED 42 is surely turned off after a lapseof the predetermined time period.

Thus, it is possible to change the exposure time period of the imagesensor 40 arbitrarily and freely by changing duration of the framestatus signal.

In addition, by changing the duration or period of the frame statussignal and the LED control signal, it is possible to arbitrarily andfreely set a light-emission period, a non light-emission period, a dutycycle of light-emission/non light-emission, etc. of the infrared-LED 42,that is, the stroboscope.

As described previously, when the reflective body 50 of thegolf-club-shaped input device 14 is irradiated by an infrared light fromthe infrared-LED 42, the image sensor 40 is exposed by the reflectedlight from the reflective body 50. In response thereto, the pixel signaldescribed above is output from the image sensor 40. More specifically,the CMOS image sensor 40 outputs an analog pixel signal shown in FIG. 7(c) in synchronous with a pixel data strobe shown in FIG. 7( b) during aperiod when the frame status signal shown in the above-described FIG. 7(a) is the high level (a period when the infrared-LED 42 is not turnedon). The game processor 52, while monitoring the frame status flagsignal and the pixel data strobe, obtains a digital pixel data throughthe A/D converter.

It is noted that the pixel data (pixel signal) is output in an order oflines such as the zero line, the first line, . . . to the thirty-firstline as shown in FIG. 8( c). It is noted that single pixel data at thehead of each of lines becomes dummy data.

Here, with referring to FIG. 9 and FIG. 10, a brief operation of thegolf game system 10 of FIG. 2 embodiment is described. A game is startedby turning the power switch 20 shown in FIG. 2 on. It is noted that thegame processor 52 shown in FIG. 5 executes an initialization process ina step S1. More specifically, the system and the respective variablesare initialized.

The initialization process in the step S1 includes a data settingprocessing with respect to the control register contained in the imagesensor 40, and is specifically executed according to the flowchart shownin FIG. 11 to FIG. 13 and at timing shown in FIG. 14.

In a first step S11 shown in FIG. 11, the game processor 52 sets acommand “CONF” as setting data. It is noted that the command “CONF” is acommand for informing the image sensor 40 of entering the setting modefor transmitting a command from the game processor 52. Then, in afollowing step S13, a command transmitting process is executed shown indetail in FIG. 12.

In a first step S31 of the command transmitting process, the processor52 sets the setting data (command “CONF” for the step S13) to theregister data (I/O port), and sets a register setting clock (I/O port)to a low level in a next step S33. Then, after a wait of a predeterminedtime period in a step S35, the register setting clock is set to a highlevel in a step S37. Then, after a wait of a predetermined time periodin a step S39, the register setting clock is set to the low level onceagain in a step S41. Thus, as shown in FIG. 14, by performing the waitof the predetermined time period, the register setting clock is changedto the low level, the high level, and the low level one after another,and whereby, a transmitting process of the command (command orcommand+data) is performed.

In a step S15 (FIG. 11), a pixel mode is set, and an exposure timeperiod is set. In the case of this embodiment, the image sensor 40 is,as described above, a CMOS sensor of 32×32, for example, and therefore,“0h” indicative of being 32×32 pixels is set to the pixel mode registerof the setting address “0”. In a next step S17, the game processor 52executes a register setting process shown in FIG. 13 in detail.

In a first step S43 of the register setting process, the processor 52sets a command “MOV”+address as a setting data, and, in a following stepS45, executes a command transmitting process described above in FIG. 12to transmit it. The processor 52 sets the command “LD”+data as thesetting data in a succeeding step S47, and executes a commandtransmitting process to transmit it in a following step S49. Theprocessor 52 sets a command “SET” as the setting data in a step S51 totransmit it in a following step S53. It is noted that the command “MOV”is a command indicative of transmitting an address of the controlregister, the command “LD” is a command indicative of transmitting data,and the command “SET” is a command for actually setting data to itsaddress. It is noted that the process is repeated in a case that thereare a plurality of control registers to be set.

Returning to FIG. 11, in a next step S19, the setting address isrendered “1” (indicative of low-Nibble address of an exposure timesetting register), and low-Nibble data “Fh” of “FFh” indicative of amaximum exposure time period is set as data to be set. Then, in a stepS21, the register setting process in FIG. 13 is executed. Similarly, ina step S23, the setting address is rendered “2” (indicative ofhigh-Nibble address of an exposure time setting register), andhigh-Nibble data “Fh” of “FFh” indicative of the maximum exposure timeperiod is set as data to be set, and the register setting process isexecuted in a step S25. Then, in a step S27, a command “RUN” indicativeof an end of the setting and for starting to output data from the imagesensor 40 is set so as to be transmitted in a step S29.

Thus, the initialization process is executed in the step S1 shown inFIG. 10. It is noted that the embodiment shown in FIG. 11 to FIG. 14 canbe changed according to a specification of the image sensor to be used.

After the step S1 in FIG. 10, the game processor 52 updates an imagesignal so as to update an image to be displayed on a monitor (not shown)in a step S2. It is noted that the update of the display image isexecuted for each frame (television frame or video frame).

Then, the game processor 52 executes a process according to a state. Itis noted that it is a selection of the game mode that has to beprocessed first. In the game mode selection, a user or a game playerselects a game mode such as one-player mode or a two-player mode byoperating the selection key 22 shown in FIG. 1, and sets a difficultylevel of the game, and so on.

In an actual golf game, it is necessary to roll a ball by swinging agolf club, and in the golf game system 10 in this embodiment, a swingingaction of the golf club is performed in a real space by use of thegolf-club-shaped input device 14. Here, the game processor 52 executes aswing determining process of the swing action to determine whether ornot the swing action is performed in a step S4. Then, when the swingaction is performed, the ball is flied or rolled in the game screen, anda ball path is calculated in a step S5. When the ball is stopped, ascore calculating and result determining process is executed in a stepS6 as a result of the ball path calculating process in the step S5.

Then, if an interruption occurs in response to a video synchronizingsignal, an image updating in the step S2 (FIG. 10) is executed.Furthermore, the sound processing in a step S7 is executed at a time ofoccurrence of a sound interruption, and whereby, a game music and asound effect such as hitting a ball, and so on are output.

With referring to FIG. 15 to FIG. 22, a specific entire operation ofsuch the golf game system 10 is described. In a first step S55 in FIG.15, the game processor 52 executes a stroboscope imaging process fordetecting a moving position of the golf-club-shaped input device 14.

A detail of the stroboscope imaging process is shown in FIG. 16. In afirst step S67 in FIG. 16, the game processor 52 substitutes “1” for thenumber of times N in the number of times register (not shown) at anarbitrary area of the internal memory (not shown). In a succeeding stepS69, the game processor 52 turns the infrared-LED 42 on for thestroboscope imaging. More specifically, the LED control signal shown inFIG. 7 is rendered the high level. Then, in a step S71, a pixel dataarray fetching process is executed.

In a first step S301 in FIG. 36, the game processor 52 sets “−1” to Xand sets “0” to Y as an element number of the image data array. Thepixel data array in this embodiment is two-dimensional array of X=0−31,and Y=0−31. However, the dummy data is output as data of a head pixel ineach line as described above, so that “−1” is set as an initial value ofX. In a succeeding step S303, a pixel data fetching process of theelements [Y] and [X] shown in FIG. 17 is executed.

In a first step S83 shown in FIG. 17, the game processor 52 checks aframe status flag signal from the image sensor 40, and in a step S85, itis determined whether or not its rising edge (from the low level to thehigh level) occurs. Then, when it is detected that the flag signal risesin the step S85, the game processor 52 instructs the A/D converterinside thereof to start to convert an input analog pixel signal todigital data in a succeeding step S87. Then, in a step S89, it isdetermined whether or not the pixel strobe from the image sensor 40 ispresent, and in a step S91, it is determined whether or not its risingedge from the low level to the high level occurs.

If “YES” is determined in the step S91, the game processor 52 thendetermines whether X=−1 or not, that is, whether the head pixel or notin a step S93. As described above, the head pixel in each line is set asa dummy pixel, so that if “YES” is determined in the step S93, withoutfetching the pixel data at that time in a next step S95, the elementnumber X is incremented in a step S97.

If “NO” is determined in the step S93, the second pixel data onward inthe line is determined, and therefore, the pixel data at that time isfetched in steps S99 and S101 so as to be stored in a temporary register(not shown). Thereafter, the process returns to a step S305 shown inFIG. 36.

In the step S305, the pixel data stored in the temporary register isstored as the elements [Y] [X] of the pixel data array.

In a following step S309, the X is incremented. If the X is less than32, the process from the step S303 to the step S307 described above isrepeatedly executed. If the X is equal to 32, that is, the fetch of thepixel data is reached to the end of the line, “−1” is set to the X in afollowing step S311 and the Y″ is incremented in a step S313 to repeatthe pixel data fetching process from the head pixel of the next line.

If the Y is 32 in a step S315, that is, if the fetch of the pixel datais reached to the end of the pixel data array, the process is returnedto a step S73 in FIG. 16.

In the step S73, the above-described pixel data array is stored asfetched data at N-th time lighting in the working area of the internalRAM, for example.

In a following step S75, the game processor 52 turns the infrared-LED 42off by rendering the LED control signal the low level, and so forth.Thereafter, in a step S76, the pixel data array when the infrared-LED 42is turned off is obtained according to a subroutine shown in FIG. 17 andFIG. 35 so as to be stored in the working area of the internal RAM in astep S77 similarly to the step S73.

Then, the number of times register N is incremented in a step S79, andit is determined whether or not the number of times N reaches apredetermined value in a step S81. If “YES” in the step S81, the processreturns to a step S57 (notable point extracting process). It is notedthat if “NO”, the process is returned to a preceding step S69.

A detail of the notable point extracting process is shown in FIG. 18. Ina first step S103 in FIG. 18, the game processor 52 substitutes “1” forthe number of times N of the number of times register (not shown) in theinternal memory (not shown). Then, in a following step S105, adifference data array is calculated from differences between the fetcheddata at lighting up and the fetched data at lighting out that arefetched in the step S73 and the step S77, respectively.

That is, in this embodiment, an infrared light is irradiated onto thereflective body 50 of the golf-club-shaped input device 14, and an imageby the reflected infrared light incident to the image sensor 40 via theinfrared filter is imaged. In a case of strobe-imaging thegolf-club-shaped input device 14 in a general room environment by use ofa general light source, in addition to the images of the reflectivebody, images of all things in the room as well as the light source suchas the fluorescent light source, an incandescent light source, sunlight(window) are imaged on the image sensor (corresponding to the imagesensor 40 in this embodiment) as shown in FIG. 19 (A). Accordingly, inorder to process the images in FIG. 19 (A) to extract only the images ofthe reflective body, a considerably high-speed computer or processor isneeded. However, in the game apparatus required to be a low cost, suchthe high-performance computer is not useable. Thus, it is necessary toperform various processes to reduce a load.

FIG. 19 (B) is an image signal when the image signal in FIG. 19 (A) issubjected to a level discrimination at a certain threshold value. Suchthe level discriminating process can be executed by a dedicated hardwarecircuit or in respect of software. However, in any method, when thelevel discrimination is executed for cutting pixel data having lightamounts less than a predetermined amount, low luminance images exceptfor the reflective body and the light source can be eliminated. In theimage shown in FIG. 19 (B), a processing of the images except for thereflective body and the light source in the room are omitted, so that,it is possible to reduce the load of the computer while the highluminance images including the light source image is still imaged, andtherefore, it is difficult to discriminate between the reflective bodyand the light source.

Here, in this embodiment, as shown in FIG. 3, by use of the infrared rayfilter 44, images except for the images by the infrared ray are notimaged on the image sensor 40. Thus, as shown in FIG. 19 (C), it ispossible to eliminate the images of the fluorescent lamp source scarcelyincluding the infrared ray. However, the sunlight and the incandescentlamp are still included in the image signal. Accordingly, in order tofurther reduce the load, the differences between the pixel data at atime of turning the infrared stroboscope on and the pixel data at a timeof turning it off are calculated.

More specifically, the differences between the pixel data of the imagedata at a time of lighting up shown in FIG. 19 (C) and the pixel data ofthe image data at a time of lighting out shown in FIG. 19 (D) arecalculated. Then, the image by the differences is obtained as shown inFIG. 19 (E). As clearly understood in comparison with FIG. 19 (A), theimage by the difference data includes only the images obtained from thereflective body of the golf-club-shaped input device 14. Accordingly,even the game processor 52 low in performance can obtain a moving pathof the reflective body 50, that is, the club head 48 (FIG. 4) by theswing of the golf-club-shaped input device 14.

For this reason, in this embodiment, the difference data array shown inFIG. 19 (E), for example, is calculated in a step S105 shown in FIG. 18.After obtaining the difference data array in the step S105, coordinatesof a pixel having the largest value (pixel having the largest luminance)is obtained in a step S107, and it is determined whether or not theluminance of the pixel at the coordinates exceeds the predeterminedvalue in a step S109.

If “YES” is determined in the step S109, it is determined whether or notpixels adjacent to the pixel at the coordinates obtained in the stepS107, and pixels further adjacent thereto exceed the above-describedpredetermined value one after another to calculate a diameter Φ [N] of anotable portion (image of the reflective body in the embodiment) in astep S111. The reason why calculating the diameter (or size) of thenotable portion is it is necessary that the height (Z coordinate) of thereflective body 50 is obtained in a step S113 and central coordinates ofthe reflective body are specified in a step S115.

As shown in FIG. 3, an optical system with a single focus is utilizedfor the imaging unit 28 of this embodiment. Therefore, when the distancebetween the reflective body 50 and the imaging device, that is, theimage sensor 40 is coincident with the focus of the above-describedoptical system, the image is reduced in blur and increased in diameter.On the contrary thereto, the further the estrangement between thefocused distance and the reflective body-image sensor distance is, thesmaller the image itself becomes. In an example of FIG. 19 (E) and FIG.21 described later, the size (diameter) of the image is changeddepending upon the distance between the reflective body and the imagesensor. Thus, the distance between the reflective body and the imagesensor, that is, the height of the reflective body 50 (Z coordinate) canbe evaluated on the basis of the diameter (size) of the image of thereflective body. Although in the golf game system of this embodiment,the Z coordinate is not utilized, by utilizing the Z coordinate asnecessary, a further different game input can be provided.

Thus, in the step S113, the Z coordinate is obtained, and in the stepS115, central coordinates (X, Y or X, Y, Z) are saved.

Then, the N value of the number of times register is incremented in astep S117, and it is determined whether or not the number of times Nexceeds a predetermined value in a step S119. If “YES” is determined inthe step S119, the process is directly returned to the step S59 in FIG.15. However, if “NO” is determined, the process is returned to thepreceding step S105 to execute the process in the step S105 onward.

It is noted that if “NO” in the step S109, that is, if it is determinedthe luminance of the pixel having the largest luminance does not exceedthe predetermined value, it is determined whether or not all thepredetermined number of data has been searched in a following step S121.If “YES” is determined in the step S121, that is, if all thepredetermined number of pixels is searched, determining that the notablepoint is absent, the result of the search is stored in a step S123. Itis noted that if “NO” is determined in the step S121, the coordinatesdata of the pixel having the next largest luminance is obtained in astep S125, and then, the process returns to the step S107.

The notable point extracting process is thus performed in the step S57,and then, a movement calculating process such as calculating thevelocity vector, and so forth is executed in a following step S59.

FIG. 20 shows a detailed embodiment of a velocity vector calculatingprocess as one example of the movement calculating process. In a stepS127 in FIG. 20, “1” is substituted for the number of times register Nas described above. Then, in a step S129, an N-th velocity vector (VX[N], VY [N]) is calculated by subtracting the (N−1)-th notable pointcoordinates (PX [N−1] PY [N−1]) from the N-th notable point coordinates(PX [N], PY [N]: FIG. 21) so as to be stored in the internal memory.

FIG. 21 shows an image of the notable area at the first time having adiameter Φ [1], the central coordinates of the notable area at the firsttime are (PX [1], PY [1]), and the central coordinates of the notablearea at the second time having a diameter Φ [2] is (PX [2], PY [2]).Similarly, the notable areas third and the fourth time have diameters Φ[3] and Φ [4], respectively, and the central coordinates thereof areindicated by (PX [3], PY [3]) and (PX [4], PY [4]), respectively.

In a case that the notable area at the second times is the N-th, the(N−1)-th notable area is the notable area at the first time. Thus, inthe step S129, the velocity vector in an X direction VX [2] is rendered(PX [2]−PX [1]), and the velocity vector in a Y direction VY [2] isrendered (PY [2]−PY [1]). It is noted that in a case of N=1, due to theabsence of the coordinates data of N−1, the velocity vector iscalculated by using final result data at the previous time, or by usingthe predetermined value in a case of the absence of final result data.

It is noted that in FIG. 21, the change amounts ΔX and ΔY of the notablearea (reflective body) of each strobe image is also illustrated.Accordingly, if necessary, a change or displacement velocity can becalculated with the use of the change amounts.

After calculating the velocity vector in the step S129, the number oftimes N is incremented in a step S131, and it is determined whether ornot the N reaches the predetermined value in a following step S133. If“NO” in the step S133, the process returns to the preceding step S129 torepeat the process in the step S129.

After the process in the step S59, a determining process described indetail in FIG. 22 is executed in a following step S61. In thedetermining process, it is determined whether or not thegolf-club-shaped input device 14 is swung. In a step S135 in FIG. 22,“1” is substituted for N, and then, in a step S137, a velocity V [N](scalar value) is calculated from the velocity vector at the N-th point(VX [N], VY [N]). Then, it is determined whether or not the velocity V[N] thus calculated exceeds the first threshold value in a followingstep S139. If “YES” in the step S139, it is directly determined theswing is performed in a step S141, and an initial velocity vector of thegolf ball on hitting is evaluated from the velocity vector at the N-thpoint. Accordingly, in a case of the golf game system in thisembodiment, a flying distance can be calculated from the initialvelocity vector, the wind direction, the strength of the wind, and thegeographical data.

If “NO” in the step S139, that is, if the swung speed of the golf clubis below the first threshold value, it is determined whether or not aline segment between the N-th point and the (N−1)-th point intersects apredetermined area in a step S145. Then, as a result of thedetermination, if “YES” is determined in a step S147, it is determinedwhether or not the speed at the N-th point (scalar value) exceeds asecond threshold value in a step 149. It is noted that the secondthreshold value is naturally a value smaller than the first thresholdvalue.

If “YES” in the step S149, the process proceeds to a preceding step S141while if “NO”, the process proceeds to the step S151 just as “NO” in thestep S147 to increment the value N. Then, it is determined whether N islarger than a predetermined value or not, and if “NO”, the processreturns to the preceding step S137 to repeatedly execute the stepsonward. It is noted that if “NO” is determined in the step S153, thatis, in a case that the line segment in the step S145 does not intersectthe predetermined region, or even if the line segment intersects thatarea, in a case the velocity is smaller than the predetermined value, itis determined that the swing is not performed in the end.

After completion of the determining process in FIG. 22, the processreturns to a step S63 shown in FIG. 15. In the step S63, a processcorresponding to the application such as the game and etc. is performed,and in a step S65, it is determined whether or not the process is ended(game over in a case of the game), and if “YES”, the process is ended.

It is noted that in the above-described embodiment, the circle-shapedreflective body 50 is provided on the golf-club-shaped input device 14,the initial velocity vector is evaluated from the moving path, andregarding that the golf ball is hit at the initial velocity, the flyingdistance of the ball is calculated. That is, the rotation applied to theball is ignored. This is because that it is impossible, if using thecircle-shaped reflective body, to specify the orientation of thegolf-club-shaped input device 14. Here, in a next embodiment, it isrendered the orientation of the golf-club-shaped input device 14 canalso be calculated.

In order to attain this, in the next embodiment, the golf-club-shapedinput device 14 shown in FIG. 23 is utilized. In this embodiment, anoval-shaped or rectangular-shaped reflective body 50A is utilized whilethe circle-shaped reflectivity body is utilized for the golf-club-shapedinput device shown in the preceding FIG. 4.

Then, after the respective pixel data at a time that the light-emittingdiode 42 lights up and pixel data at a time the LED 42 lights out arefetched in the steps S55 of FIG. 15, the notable point extractingprocess shown in FIG. 24 is executed. It is noted that the stepsS157-S163 are the same as the steps S103-109 in FIG. 18.

Then, in a following step S165, it is determined whether or not pixelsadjacent to the pixel of the coordinates obtained in the step S161 and apixel adjacent to the pixel exceed the predetermined value one afteranother to extract all the pixels at the notable point (image of thereflective body in this embodiment). Then, in a step S166, two pixels ofPa (Xa, Ya) and Pb (Xb, Yb) that are the farthest from each other areextracted out of all the pixels of the notable point. The two points, asshown in FIG. 26, indicate both ends of the rectangular-shapedreflective body 50A in the direction of the length. There are no pointsthat are the farthest from each other except for the both ends of thelong side.

Then, in a step S167, middle point coordinates between the two points Paand Pb are stored as the coordinates at N-th point (PX [N], PY [N]) inthe memory. Then, in a step S169, a tilt between the Pa and the Pb iscalculated so as to be stored as angle data θ [N] as shown in FIG. 26.It is noted that the tilt θ [N] is, as shown in the step S169,calculated as an arc tangent of (Xa−Xb)/(Ya−Yb).

Thus, the direction of the golf-club-shaped input device 14 with respectto the imaging device is obtained as the angle data θ [N].

The steps S171-S179 in FIG. 24 respectively are the same as the stepsS117-S125 in FIG. 18.

Then, in the next determining process, the respective steps shown inFIG. 25 are executed. However, steps S181-S189 and steps S191-steps S201are the same as the steps S135-S143 and the steps S145-S155 shown inFIG. 22, respectively.

In a step S203 in FIG. 25, the angle θ j (FIG. 26) is calculated by thecoordinates at N-th point and the coordinates at (N−1)-th point. Theangle θ j can be calculated by the arc tangent (PY [N]−PY [N−1])/(PX[N]−PX [N−1]) as shown in the step S203. In a step S205, a tilt θ k (=θ[N]−θ j) of the golf-club-shaped input device 14 with respect to thedirection of the swing of the golf-club-shaped input device 14 iscalculated according to an equation shown in the step S205, and ahook/slice parameter on hitting is calculated from the θ k. Thus, byobtaining the hook/slice parameter, changing a flying direction due tothe spin of the ball as well as simply changing the flying distance canbe added, and therefore, it is possible to provide a more reality andinterest to the golf game system of this embodiment.

With referring to FIG. 27, a bowling game system 78 of anotherembodiment according to this invention includes a game machine 80, andthe game machine 80 is driven by an AC adaptor or a battery, andconnected to an AV terminal (not shown) of a television monitor (notshown) through an AV cable 16 similarly to the game machine 12 in thegame system in FIG. 2.

The game machine 78 further includes a housing 82, and on the housing82, a power switch 84 is provided, and a direction button 86, adetermination key 88, and a cancel key 90 are also provided. Thesebuttons or operation keys have the same function as corresponding onesin FIG. 2.

A part of the housing 82 of the game machine 80 is partly cut away, andat that place, a movable body 92 is rotatably supported in an elevationdirection. On the side surface of the movable body 92, the imaging unit28 described with referring to FIG. 3 is housed, and therefore, themovable body 92 is provided with an image sensor 40 similarly to theabove-described one. Then, an infrared-LED 42 is provided on the sidesurface of the movable body 92 in proximity to the image sensor 40 andin an integrated manner with the image sensor 40 so as to be moved inthe elevation direction, and constitutes a stroboscope imaging means.

The movable body 92 is supported such that it has constant degree offreedom in the elevation direction in this embodiment. However, thedegree of freedom may be provided in a revolution direction in place ofthe elevation direction, or in addition to the elevation direction.Specifically, the movable body 92, that is, the image sensor 40 and theinfrared-LED 42 are provided changeable in an arbitrary direction.

It is noted that if a wider angle lens is used as the lens of the imagesensor 40 (concave lens and convex lens in FIG. 4), the image sensor 40needs not to be movable, and alternatively fixedly attached to the imagesensor 40.

A ball-shaped input device 94 has holes 94 a, 94 b, and 94 c into whichthree fingers of the user, that is, a thumb, a middle finger, and a ringfinger are respectively inserted as does in the real bowling game, andis provided with a wider hole 94 d into which a child can insert any oneor a plurality fingers except for the thumb. Then, a strap 96 isprovided, and therefore, the game player equips it in his arms (upperarm or front arm) in order to assure the safety of the player. That is,the ball-shaped input device 94 is coupled to his arm by the strap 96,and therefore, even if the game player releases the ball-shaped inputdevice 94 from his fingers as does in the real bowling game, it ispossible to prevent accidents of the ball-shaped input device 94 beingflown away, being hit against the player himself or others, and soforth.

Furthermore, the ball-shaped input device 94 in this embodiment formsthe housing of the ball-shaped input device 94 by connecting transparentor translucent hemisphere outer shells 98A and 98B to each other bybosses, and inside the hemisphere outer shells 98A and 98B, hemisphereinner shells 100A and 100B that are similarly coupled to each other bybosses are fixed. Then, a reflective sheet is pasted on the surface ofthe respective hemisphere inner shells 100A and 100B to form areflective body. That is, the inner shells become the reflective body.Accordingly, the reference numeral 100 is assigned to the reflectivebody in this embodiment.

Furthermore, in the bowling game system 78 in this embodiment, themovement of the ball-shaped input device 94 is detected by thestroboscope, and whereby, a position of a bowling ball 104 is controlledon a game screen 102 as shown in FIG. 29. The game screen 102 isdisplayed as a perspective image viewed from a viewpoint of the user orthe player. That is, on the game screen 102, a bowling lane 106 and pins108 arranged in a depth direction are displayed. On the game screen, thebowling ball 104 is moved on the lane 106, and in correspondence to areached position and intensity, the pins are knocked down as do in thereal bowling game. It is noted that if from directly before the bowlingball 104 hit the pins 108, an enlarged image of the pins portion isdisplayed on the window (not shown) at the center of the screen, forexample, so much presence can be provided to the players.

It is noted that every time that the player finish throwing the ball,the game screen 102 shown in FIG. 29 and a score sheet shown in FIG. 30are switchably displayed. Then, in a case that a plurality of gameplayers play the game, the scores of the respective players aresimultaneously displayed. FIG. 30 example shows an example in which fourgame players simultaneously go in the bowling game.

In the bowling game system 78, when the player performs an throwingaction in a real space by use of the ball-shaped input device 94, thegame processor 52 (FIG. 31) intermittently turns the infrared-LED 42 on,and intermittently detects the positions of the ball-shaped input device94 by analyzing and processing the images of the CMOS image sensor 40 ateach lighting-up time and lighting-out time. Then, in correspondence tothe positions (coordinates) of the ball-shaped input device 94, themovement of the bowling ball 104 is controlled to knock down zero ormore pins.

With referring to FIG. 31, the ball-shaped input device 94 as describedabove is irradiated by a light emission from the infrared-LED 42, andreflects the infrared ray by the reflective body 100. The reflectedlight from the reflective body 100 is imaged by the CMOS image sensor40, and whereby, an image signal of the reflective body 100 is outputfrom the CMOS image sensor 40. It is noted that the other part is thesame as the golf game system 10 shown in FIG. 5.

Herein, with referring to FIG. 32, an operation of the bowling gamesystem 78 in this embodiment is schematically described. After a powerswitch 84 is turned on to start a game shown in FIG. 27, the gameprocessor 52 shown in FIG. 31 first executes an initializing process ina step S1. More specifically, systems and respective variables areinitialized. It is noted that a detailed method of the initialization isdescribed above.

Then, after the step S1 in FIG. 32, the game processor 52 updates animage signal to update an image to be displayed on the monitor 20 in astep S2. It is noted that the update of the display image is executedevery one frame (television frame or video frame).

Then, the game processor 52 executes a process depending on the state(status). It is noted that a process to be executed first is a game modeselection. In the game mode selection, the user or the game playerselects the game mode such as an one-player mode, a two-player mode,etc. by operating the selection key 86 shown in FIG. 27, and sets adifficulty level of the game in a step S3 in FIG. 32.

Although in the real bowling game, it is necessary to throw the ball onthe lane, in the bowling game system 10 in this embodiment, a throwingaction is performed by use of the ball-shaped input device 94 asdescribed above. Here, the game processor 52 executes a throwingdetermining process to determine whether or not the throwing action isperformed in a step S4. Then, if the throwing action is performed, whenthe ball 104 moves on the lane 106 (the both in FIG. 30), the ball pathis calculated and a hitting determining process of the ball 104 againstthe pins 108 (FIG. 30) is executed in a step S5. Then, when the ball 104reaches the end of the lane 106, a score calculating and resultdetermining process is executed in a step S6 as the result of the pinhitting determining process in the step S5.

It is noted that in the bowling game system 78 in this embodiment, byimaging the reflective body 100 with the stroboscope, the game input isperformed, and this is the same as the preceding embodiment.Accordingly, this embodiment is different from the preceding embodimentin only the determining processing in the step S61 in FIG. 15.

The throwing determining process step is shown in detail in FIG. 33, andan N value is set to “1” in a first step S207. Then, in a step S209, itis determined whether or not a Y component of the velocity vector(component in up and down directions) VY [N] (FIG. 21) exceeds apredetermined value. If “YES” in the step S209, the process proceeds toa step S211 to increment a counter formed in an internal memory, forexample. Then, in a next step S213, it is determined whether or not acount value of the counter is a predetermined constant C (for example,“3”). If “YES”, it is determined that a throwing action is performed ina step S215. In a succeeding step S217, the X coordinate PX [N] at theN-th point is rendered as the X coordinate at the throwing position, andin a step S219, an initial velocity of the ball on throwing is evaluatedfrom the velocity vector at N-th point (VX [N], VY [N]). Then, accordingto the initial velocity, regarding that the bowling ball is thrown, thegame processing is executed.

It is noted that if “NO” in the step S209, the above-described counteris reset in a step S221. Accordingly, the throwing action is notdetermined until the Y component of the velocity vector exceeds thepredetermined value successive three times, for example. Thus, it ispossible to prevent an undesired action of the game player from beingreflected on the game.

After the step S221, the N value is incremented in a step S223, and itis determined whether or not the N value reaches a predetermined valuein a step S225. If “YES” is determined, it is determined that thethrowing is not performed in a step S227. Then, the process returns tothe step S63 in FIG. 15 similarly to the step S219 onward.

FIG. 34 shows another embodiment of an input device utilizing astroboscope. The input device 112 of this embodiment is a glove-shapedinput device. The glove-shaped input device 112 includes gloves 114L and114R to be attached to both right and left hands, and reflective bodies116L and 116R in the form of the reflective sheet are provided atpredetermined positions (the tip end portion) of the gloves 114 l and114R. These reflective bodies 116L and 116R are respectively formed as apart of the gloves 114L and 114R, and alternatively may be pasted on thegloves 114L and 114R.

Then, in order to apply an input signal, the user puts on the gloves114L and 114R, and moves both hands over the imaging unit 28 (FIG. 3) ofthe game machine 80 shown in FIG. 27, for example. Then, according tothe step S55 in FIG. 15 described above or FIG. 16, both of thereflective bodies 116L and 116R are irradiated by the infrared-LED 42,or not irradiated, and imaged by the image sensor 40. Then, according tothe step S57 in FIG. 15, that is, FIG. 18, a notable point (tworeflective bodies 116 and thus two notable points in this embodiment) isextracted. Thereafter, an action calculating or detecting process isexecuted by applying the process in the step S59 in FIG. 15. It is notedthat in a case of utilizing the glove-shaped input device 112 in FIG.34, a modification is applied to the step S59 as shown in FIG. 35.

In a flowchart shown in FIG. 35, a moving average is detected orcalculated to obtain an input. Describing in detail, in a first stepS207, a “predetermined value −M” is set to the number-of-times registerN. Next, in a step S209, “0” is set into the variables Σ X and Σ Y.

In a step S211, coordinates (PX [N], PY [N]): FIG. 21) at N-th time areobtained. It is noted that in a case of N<1, coordinates data isobtained from coordinate information of the last time. In a succeedingstep S213, the coordinates obtained in the step S211 are respectivelyadded to the variables Σ X and Σ Y initialized in the step S209 toupdate the variables Σ X and Σ Y. This process is repeated until it isdetected in a step S217 that the number of times N incremented in thestep S215 reaches the predetermined value. Accordingly, the gameprocessor 52 stores, at this time, the variables Σ X and Σ Y to which Msets of coordinates are added. Then, in a step S219, each of thevariables Σ X and Σ Y is divided by the number M to calculate the movingaverage (AX, AY). By use of the moving average AX, AY, the gameprocessor 52 changes positions of moving objects that can be operated bythe player on the game screen.

It is noted that the present invention can be embodied or modifiedexcept for the above-described embodiments.

For example, in the embodiment shown in FIG. 2, the input is performedby the use of the golf-club-shaped input device 14. However, byutilizing the similar system, a number of modifications are conceivableincluding a baseball game apparatus by use of a bat-shaped input devicefor baseball and/or a ball-shaped input device for baseball, a tabletennis game by use of a racket-shaped input device for table tennis, atennis game by use of a racket-shaped input device for tennis, etc.

Furthermore, a modification of soccer game apparatus is alsoconceivable, in which by attaching an input device provided with areflective body around the part of the lower leg or the ankle, aposition, velocity, a pattern of a moving path of the leg of the playerare used as an input signal.

In addition, as to the embodiment utilizing the glove-shaped inputdevice shown in FIG. 34, a moving average value calculated in theflowchart in FIG. 35 is utilized as an input signal. Here, modificationsincluding a boxing game apparatus and a dancing game apparatus areconceivable, in which a position, a moving velocity, a moving path ofthe glove-shaped input device are calculated to be used as an inputsignal. Furthermore, in these modified examples, in place of theglove-shaped input device, a wristband shaped input device wrappedaround the wrist secures the same effect as the glove-shaped inputdevice.

A further modification including a dancing game apparatus utilizing thearms and the legs is conceivable, in which the above described inputdevice wrapped around the leg is used in combination with theglove-shaped input device shown in FIG. 34 or the wristband shaped inputdevice.

In addition, other modifications including a sword battle game apparatusare conceivable, in which like the golf-club-shaped input device 14shown in FIG. 23, the long-thin shaped reflective body 50A is pasted onthe input device in the form of a sword, and an angle, a position, amoving velocity, a moving path of the sword are used as an input device.

According to this invention, an object is imaged by use of thestroboscope and the imaging means, and on the basis of the differencebetween the plurality of image signals at a time of lighting up and animage signal at a time of lighting out, a part or all of the informationof the position, the size, the velocity, the acceleration, and themoving path pattern are calculated. The information processing apparatusand an entertainment apparatus execute the information processing, thegame, and other entertainment processing by use of the information.

It is noted that although all the information processing is executed bya single game processor in the above-described embodiment, entireprocessing can be shared with the use of two or more processors orcomputers.

Although the present invention has been described and illustrated indetail, it is clearly understood that the same is by way of illustrationand example only and is not to be taken by way of limitation, the spiritand scope of the present invention being limited only by the terms ofthe appended claims.

1. An information processing apparatus provided with an input systemutilizing a stroboscope, comprising: a stroboscope including a lightsource outputting a light of a specific wavelength range; a base unitwhich includes a supporting cylinder having an opening and a lensprovided below said opening in said supporting cylinder; a filterprovided so as to cover said opening of said supporting cylinder andpassing only the light of said specific wavelength range; a first objectincluding a first retroreflective body; said light source being providedin the vicinity of said filter so as to light up said first object; animager, which is provided within said base unit and below said lens,imaging said first object at a light-emission and at anon-light-emission of said stroboscope to produce a plurality of imagesignals at light-emission and a plurality of image signals at non-lightemission; a calculator calculating a part or all of information of aposition, a size, a velocity, an acceleration, a moving path pattern ofsaid first object by detecting a first notable portion in correspondenceto said first retroreflective body from a difference between said imagesignals at light-emission and said image signals at non-light emission;and an information processor performing information processing accordingto an application on the basis of the information calculated based onsaid difference by said calculator to provide an output according tosaid application.
 2. An information processing apparatus according toclaim 1, wherein said calculator includes a determiner determiningwhether or not said information based on said difference is coincidentwith a predetermined condition, and further includes a valid inputtransmitter transmitting said information based on said difference tosaid processor as the valid information being performed only if saiddeterminer determines that said information based on said differencesatisfies said predetermined condition.
 3. An information processingapparatus according to claim 1, wherein said calculator includes adistance calculator extracting said first notable portion from saiddifference and calculating a distance between said first object and saidimager from the information indicative of a size of said first notableportion.
 4. An information processing apparatus according to claim 1,wherein said calculator includes an extractor extracting a shape of saidfirst notable portion from said difference between said image signal atlight-emission and said image signal at non-light emission, and a firstangle calculator calculating a first angle indicative of an inclinationof said shape corresponding to said first object.
 5. An informationprocessing apparatus according to claim 4, wherein said extractorextracts predetermined two points within said first notable portion, andthe calculation of the first angle by said first angle calculator is forcalculating an angle between a line segment between the predeterminedtwo points and a predetermined coordinate axis.
 6. An informationprocessing apparatus according to claim 4, wherein said calculatorincludes a swing direction calculator calculating a swing direction ofsaid first object on the basis of a movement of said first notableportion, and a second angle calculator calculating a second angleindicative of an inclination of said first object with respect to saidswing direction on the basis of said swing direction calculated by saidswing direction calculator and said first angle calculated by said firstangle calculator.
 7. An information processing apparatus according toclaim 6, wherein said information processor includes a parametercalculator calculating a parameter of a hook/slice in hitting a ball onthe basis of said second angle.
 8. An information processing apparatusaccording to claim 1, wherein a time interval of the light-emission ofsaid stroboscope is freely settable.
 9. An information processingapparatus according to claim 1, wherein a length of the light-emissionand a length of the non-tight-emission of said stroboscope are freelyconfigurable.
 10. An information processing apparatus according to claim1, wherein an exposure period of said imaging means is freelyconfigurable.
 11. An information processing apparatus according to claim1, wherein said imager includes an imaging device for imaging only saidlight having the specific wavelength range.
 12. An informationprocessing apparatus according to claim 1, wherein each of saidcalculation of said calculator and said information processing of saidinformation processor are processed by a single or a plurality ofprocessors.
 13. An information processing apparatus according to claim1, wherein the information processing performed by said informationprocessor is an entertainment processing such as a game, etc.
 14. Aninformation processing apparatus according to claim 1, wherein saidinformation processor includes a moving object controller controlling amovement of a moving object displayed on a screen based on saidinformation calculated based on said difference by said calculator. 15.An information processing apparatus according to claim 14, wherein saidcalculator includes a determiner determining whether or not saidinformation based on said difference is coincident with a predeterminedcondition, and said moving object controller gives a change on saidmoving object in said screen if said determiner determines that saidinformation based on said difference satisfies said predeterminedcondition.
 16. An information processing apparatus according to claim15, wherein said determiner determines whether said first object wasswung on the basis of the speed of said first object.
 17. An informationprocessing apparatus according to claim 16, wherein said determinerdetermines that said first object was swung when the speed of said firstobject becomes equal to or larger a predetermined value in a pluralityof succeeding times.
 18. An information processing apparatus accordingto claim 16, wherein said moving object controller calculates an initialspeed of said moving object on said screen on the basis of the speed ofsaid first object calculated by said calculator if and when saiddeterminer determines that said first object was swung.
 19. Aninformation processing apparatus according to claim 18, wherein saidmoving object on said screen is an image of a ball for golf game.
 20. Aninformation processing apparatus according to claim 14, wherein saidmoving object controller controls a movement of said moving object ofsaid screen on the basis of the position of said first object calculatedby said calculator.
 21. An information processing apparatus according toclaim 14, wherein said moving object controller calculates an initialspeed of said moving object on said screen on the basis of the speed ofsaid first object calculated by said calculator.
 22. An informationprocessing apparatus according to claim 14, wherein said informationprocessor includes a determiner determining whether said moving objecton said screen collides with a predetermined image on said screen. 23.An information processing apparatus according to claim 22, wherein saidmoving object on said screen is an image of a ball for a bowling gameand said predetermined image on said screen is an image of a pin for thebowling game.
 24. An information processing apparatus according to claim1, further comprising a movable body rotatable in at least one of anelevation direction and a revolution direction, said imager, base unitand light source being accommodated in said movable body.
 25. Aninformation processing apparatus according to claim 1, wherein saidfirst retroreflective body is provided in a transparent orsemitransparent housing.
 26. An information processing apparatusaccording to claim 1, wherein said first object is provided with astrap.
 27. An information processing apparatus according to claim 1,wherein said first object is attached to a lower leg or an ankle of aplayer.
 28. An information processing apparatus according to claim 27,wherein said application is for a soccer game.
 29. An informationprocessing apparatus according to claim 1, wherein said first object isin a form of a band.
 30. An information processing apparatus accordingto claim 1, wherein said first object is in a form of a glove.
 31. Aninformation processing apparatus according to claim 1, wherein saidfirst retroreflective body has a long-thin shape, and said calculatordetects said first notable portion from said difference=and calculatesat least one of an angle, position, moving speed and moving path patternof said first object, and said information processor performs a swordbattle game on the basis of said at least one of the angle, position,moving speed and moving path pattern of said first object calculated bysaid calculator.
 32. An information processing apparatus according toclaim 1, further comprising a second object including a secondretroreflective body, wherein said imager images said second object at alight-emission and at a non-light-emission of said stroboscope toproduce a plurality of image signals at light-emission and a pluralityof image signals at non-light emission; said calculator calculates apart or all of information of a position, a size, a velocity, anacceleration, a moving path pattern of said second object by detecting asecond notable portion in correspondence to said second retroreflectivebody from difference between said image signal at light-emission andsaid image signal at non-light emission; and said information processorperforms information processing according to an application on the basisof the information of the first object and the second object calculatedbased on said difference by said calculator to provide an outputaccording to said application.
 33. An information processing apparatusaccording to claim 32, wherein said first object is attached to a handor wrist, or held by a hand of a player, and said second object isattached to a lower leg or ankle of said player.
 34. An informationprocessing apparatus according to claim 33, wherein said application isfor a dance game.
 35. An information processing apparatus according toclaim 1, wherein said application is for any one of a bowling game,baseball game, table tennis game, tennis game, soccer game, boxing game,dance game and sword battle game.
 36. A computer readable medium storinga program for an information processing apparatus provided with an inputsystem utilizing a stroboscope which includes a light source outputtinga light of a specific wavelength range; a base unit which includes asupporting cylinder having an opening and a lens provided below saidopening in said supporting cylinder; a filter provided so as to coversaid opening of said supporting cylinder and passing only the light ofsaid specific wavelength range; a first object including a firstretroreflective body, wherein the light source is provided in thevicinity of said filter so as to light up said first object; and animager which images the light passing through said filter, said programcausing a processor of said information processing apparatus to execute:an imaging step of imaging by said imager said first object at alight-emission and at a non-light-emission of said stroboscope toproduce a plurality of image signals at light-emission and a pluralityof image signals at non-light emission; a first step of calculating apart or all of information of a position, a size, a velocity, anacceleration, a moving path pattern of said first object by detecting afirst notable portion in correspondence to said first retroreflectivebody from a difference between said image signal at light-emission andsaid image signal at non-light emission; and a second step of performinginformation processing according to an application on the basis of theinformation calculated based on said difference by said first step toprovide an output according to said application.